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EVALUATING AN ORGANIZATION'S LEVEL OF SELF -REPORTING 

CROSS-REFERENCE TO RELATED APPLICATIONS 
This application claims the priority of U.S, Provisional 
5 Patent Application Serial No. 60/307,482, filed on July 24, 
2 0 01, which is incorporated herein by reference. 

BACKGROUND 

The disclosure relates to evaluating an organization's 

10 level of self -reporting. 

Executives often find themselves trying to manage 
expectations about their organization's earnings. As a 
result, some companies may disclose information about the 
company required by regulation, but little non- financial 

15 information that investors and other stakeholders seek. For 
example, in the context of a publicly traded company, the 
information disclosed may reveal little about future stock 
price performance and may lead to excessive stock price 
volatility, inaccurate valuations and over— reliance on market 

20 gossip. Adequate information about intangible assets and non- 
financial value drivers-which can serve as leading indicators 
of future financial success— are often missing from such 
traditional financial reporting. 

25 SUMMARY 

A software-based tool provides an evaluation of a 
company's level of reporting about itself. The tool can 
provide an assessment of a company' s communications against a 
framework that assumes transparent reporting is desirable. A 

30 comparison with pre-selected criteria, such as a pre-selected 
peer group of companies or a set of recommended practices may 
be provided in some implementations. Based on the results of 
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the analysis, a score can be generated for each area of the 
framework. In some implementations, the scores may be 
summarized in an executive level presentation that can include 
benchmark results against the framework and the pre -selected 
5 criteria, identification of best practice examples, and 
recommendations for improvement. 

In one aspect, a method includes entering information in 
a computer system with respect to performance measures 
indicative of an organization's level of reporting about 

10 itself and causing the computer system to generate an 

assessment of the organization's level of reporting based on 
the received information. 

According to some implementations, publicly available 
sources of an organization's external communications are 

15 examined. Information is entered on a questionnaire with 

respect to performance measures based on the examined sources 
to provide an indication of the organization's level of 
external reporting about itself. An assessment of the 
organization's level of external reporting is received from a 

20 computer system based on the information entered on the 
questionnaire . 

The detailed description also discloses an apparatus that 
includes a database to store templates for one or more 
questionnaires for use in connection with scoring performance 

25 measures based on an organization's level of reporting about 
itself. A processor is coupled to the database. Memory 
includes instructions that, when applied to the processor, 
cause the processor to provide a questionnaire based on the 
templates, and to generate an assessment of the particular 

30 organization's level of reporting about itself based on 
information entered on the questionnaire by a user. 
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The techniques, described in greater detail below, can 
help an organization become more informed about the extent and 
types of information it disseminates to the public about 
itself. An assessment of how well the organization performs 
5 can help the organization address deficiencies in its external 
reporting . 

The techniques also can be used to assist the 
organization in understanding how well it communicates 
information to employees, management and other stakeholders 
10 within the organization. 

Other features and advantages will be readily apparent 
from the following detailed description, the accompanying 
drawings and the claims. 

15 BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a block diagram of a system that includes a 
tool for evaluating an organization's level of reporting. 

FIGS. 2A though 2E illustrate portions of a questionnaire 
for use in the evaluation. 
20 FIG. 3 is a flow chart of a method of evaluating an 

organization's level of reporting. 

FIG. 4 illustrates an example of a completed 
questionnaire . 

FIG. 5 is a chart showing examples of communication types 
25 and the corresponding number of points that are awarded in one 
implementation. 

FIG. 6 is a chart illustrating an example of calculating 
total scores for a performance measure. 

FIG. 7 is a radar diagram comparing an organization's 
30 score against recommended practices. 
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DETAILED DESCRIPTION 
As shown in FIG. 1, a system includes an evaluation tool 
hosted, for example, on a server 10 that can be accessed from 
a personal computer 12 over the Internet or other network 14 . 
5 A database 16 is associated with the server 10 and stores 
templates for generating questionnaires 22. Results of the 
evaluation can be stored in another database 18. The results 
can be displayed and subsequently presented to the evaluated 
organization. In some implementations, the evaluated 

10 organization may be given access to the results through an 
Extranet or through on-line subscription rights. 

As shown in FIGS. 2A through 2E, in one particular 
implementation, the organization is scored against a framework 
that includes the following categories: Market Overview (FIG. 

15 2A) , Value Strategy (FIG. 2B) , Managing for Value (FIG. 2C) 
and Value Platform (FIGS. 2D and 2E) . The Market Overview 
category relates to management's assessment of the company's 
competitive position, assumptions about the macro -economic 
environment and industry growth, views on regulatory 

20 environment and perceptions about current and future 

technologies. The Value Strategy category relates to the 
company' s overall corporate strategy and its strategies for 
major business units, as well as how the company intends to 
implement those strategies in terms of organization and 

25 governance, structures and processes. The Managing for Value 
category relates to the measures that the company believes 
most closely reflect determinants of and changes in 
shareholder value. The Value Platform category relates to 
information on non- financial value drivers such as innovation, 

30 intellectual capital, customers, brands, supply chain, people 
and reputation. 
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Each category in the framework has one or more elements 
each of which has a respective suite of performance measures 
associated with it. In this example, the performance measures 
serve as predictive indicators of future shareholder value 
5 creation. In general, the performance measures represent 
information that may be used by management, investors, 
analysts and others to gain an understanding of the 
organization's performance in financial and non-financial 
areas . 

10 In the example illustrated by FIG. 2A, the category 

Market Overview includes the elements Competitive Environment, 
Regulatory Environment and Macro-Economic Environment. In 
this example, the element Competitive Environment relates to 
external constituents and dynamics that impact the current or 

15 future business environment, including customers, suppliers, 

competitors, globalization and new technologies. That element 
has the following performance measures: Market Growth, Level 
of Current and Future Competition, Industry and Business 
Outlook and Industry and Business Outlook (by segment) . The 

20 performance measure Market Growth, for example, refers to the 
increase in size of the total market as defined by the 
organization. The elements and performance measures in the 
questionnaire illustrated in FIGS. 2A through 2E are intended 
as examples . 

25 The database 16 stores templates for questionnaires to be 

used with the evaluation tool. To initiate an evaluation, a 
user accesses the evaluation tool, for example, from the 
personal computer 12. In some implementations, the evaluation 
tool may be accessed through a web page. After accessing the 

30 evaluation tool, the user enters information about the 

organization to be evaluated in response to prompts from the 
evaluation tool. The evaluation tool generates questionnaires 
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based on the templates in the database 16 and the information 
provided by the user. The questionnaires are sent to the user 
for display on the personal computer 12. 

One implementation uses the following three 
5 questionnaires: an Annual Report Questionnaire (ARQ) , an 
Investors Briefing Questionnaire (IBQ) and an Other Media 
Questionnaire (OMQ) . In this particular example, the 
questionnaires are designed to capture information reported 
externally by the organization. For example, the ARQ 

10 identifies information obtained from the organization's annual 
report. Portions of the ARQ are illustrated in FIGS. 2A 
through 2E. The IBQ identifies information from presentations 
and reports to analysts or investors, from speeches and from 
question and answer sessions held by the organization. 

15 Similarly, the OMQ identifies information from environmental 
reports, social impact reports, press releases and the 
organization's website. The IBQ and OMQ can have a format 
similar to the format of the ARQ shown in FIGS. 2A-2E. 

In other implementations, the questionnaires can be 

20 designed to help determine the level of the organization' s 

reporting about itself to its employees or other stakeholders. 

Some implementations allow the user to add or delete 
elements and performance measures from the questionnaires. 
Thus, the questionnaires may be tailored to the particular 

25 organization that is to be evaluated. 

Information reported by the organization about itself may 
be presented qualitatively through a narrative description or 
quantitatively through the use of numbers, statistics, 
percentages, graphs, etc. As illustrated by FIGS. 2A-2E, the 

30 questionnaires list six ways— or communication types— in which 
information may be presented: 1. Qualitative information (QL) ; 
2. Quantitative information for the current period (QN-C) ; 3. 



-6- 



Attorney Docket No. 13325-002001 



Quantitative information for a prior period (QN-PP) ; 4. 
Benchmarking information (QN-BM) ; 5. Quantitative target 
information for the current period (QN-CT) ; and 6. 
quantitative target information for a future period (QN-FT) . 
5 The second through sixth communication types are represented 
by quantitative information. 

As indicated by FIG. 3, as part of the evaluation 
process, one or more persons, referred to as scorers, examine 
100 all relevant available sources of reporting by the 

10 organization and complete 102 the questionnaires. For 

example, if the goal of the evaluation is to determine the 
organization's level of reporting about itself to the public, 
publicly available sources of external information by the 
organization would be examined. Information about the various 

15 performance measures listed in the questionnaires is 
identified. If information relating to a particular 
performance measure is disclosed in the examined sources, the 
scorer enters ^^YES" in the appropriate box on the 
questionnaire. If the scorer does not find any information 

20 for a particular communication type, then '''NO" is entered in 
the appropriate box. Preferably, data for a performance 
measure that is not explicitly mentioned in the organization's 
reporting should not receive a positive score even though the 
data can be calculated from the other disclosed information. 

25 Communication types that are inapplicable for a 

particular performance measure are blocked out on the 
questionnaire and need not be scored. In the illustrated 
implementation, for example, a benchmark for the performance 
measure Market Growth listed under the element Competitive 

30 Environment in the category Market Overview is inapplicable 
and, therefore, has been blocked out (FIG. 2A) . 
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According to one implementation, the ARQ and IBQ should 
be completed before the OMQ. When the scorer accesses the 
OMQ, the evaluation tool automatically indicates which 
performance measures received a non-zero score during 
5 completion of the ARQ and IBQ. Thus, the scorer need only 

address the remaining performance measures when completing the 
OMQ. For example, a press release may explain the company's 
strategy which also was disclosed in the company' s annual 
report. In that case, no additional score would need to be 

10 entered on the OMQ in connection with the corresponding 
performance measure . 

As shown, for example, in FIG. 2A, each questionnaire 
includes a column (''Reference'') to allow the scorer to list or 
cross-reference the source of the data. Preferably, two-way 

15 referencing should be used. The specific source of the 
information that serves as the basis for the score can be 
listed in the Reference column. Additionally, the 
questionnaire, the performance measure and the communication 
type(s) that were awarded a non-zero score can be noted on the 

20 document itself. 

As also shown, for example, in FIG. 2A, each 
questionnaire includes a Comments column that allows the 
scorer to provide additional comments. One group of comments- 
permanent comments— may specify what information was 

25 communicated in the examined sources as well as 

recommendations for improvement. A second group of comments- 
transitory comments— can relate to issues that need to be 
addressed with other members of a team assisting in the 
evaluation. The different types of comments can be entered in 

30 separate fields of the Comments column. The information in 
those columns can be used to confirm the scoring is accurate 
and to facilitate quality control. The evaluation tool can 
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delete transitory comments automatically from the 
questionnaires after they have been reviewed and addressed. 

FIG. 4 illustrates an example of a completed 
questionnaire . 

5 Where quantitative information is provided in the 

company's reports for only a specific sector or geographical 
segment of the business, such information may be considered 
sufficient to generate a non-zero score for the relevant 
performance measure. In that situation, comments can be 

10 provided in the Comments column (e.g., FIG. 2A) to indicate 
the proportion of the business for which the information was 
provided. A comment also can be added recommending that such 
data be provided for all sectors of the business. 

After entering the information on the questionnaire (s) , 

15 the user would, for example, click an appropriate graphical 
user interface on the computer screen associated with the 
computer 12 to cause the evaluation tool to perform the 
evaluation. The evaluation tool automatically awards 104 
(FIG. 3) a score for each performance measure in the 

20 questionnaires based on whether the performance measure is 

communicated in one or more of the six defined communication 
types in the source being reviewed. FIG, 5 lists the number 
of points that are awarded for each communication type 
according to one implementation. The number of points awarded 

25 for a particular performance measure and communication type is 
the same regardless of whether the same type of information 
appears only once or more than once in the organization' s 
external reporting . 

Typically, quantitative information is accompanied by 

30 qualitative information in the form of narrative. Therefore, 
in some implementations, the evaluation tool automatically 
generates a qualitative score for a particular performance 
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measure when a non-zero score is entered for a non- qualitative 
communication type with respect to the same performance 
measure . 

In the illustrated example, the maximum score that the 
5 organization can receive in connection with a particular 

performance measure is ''10.'' That score would be awarded if 
the organization's reporting disclosed information in each of 
the communication types in connection with a particular 
performance measure . 

10 Once the performance scores are entered into the 

questionnaires, the evaluation tool automatically calculates a 
total score for each element in the framework with respect to 
each of the communication types. 

A quality control process can be used to help assure that 

15 each organization is scored accurately and consistently. In 
one implementation, the quality control process includes three 
levels of review: scorer review, engagement review and core 
team review. At each level of review, the evaluation tool 
automatically generates 106 (FIG. 3) an exception report. 

20 Exceptions may be generated, for example, if a 

quantitative score is obtained for a ''stretch measure.'' A 
stretch measure refers to a performance measure for which 
there is no general agreement as to how that performance 
measure should be calculated. Examples of stretch measures 

25 include human capital, quality of management and corporate 
citizenship. An exception is generated if a quantitative 
score is provided for such a performance measure. 

An exception also may be generated with respect to 
performance measures required by international accounting or 

30 other standards, but for which no score was generated. 

Similarly, an element in the framework having a total score of 
zero will cause an exception to be generated. Additionally, 
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an exception can be generated if the score for a particular 
framework element falls outside an expected range, for 
example, if the score is unexpectedly high or low. In 
particular implementations, additional or different exceptions 
5 may be generated automatically by the evaluation tool. 

Once the questionnaires and the quality review process 
are completed, the evaluation tool generates 108 (FIG. 3) 
analysis results based on the received information. A total 
score or rating indicative of the organization's level of 

10 reporting about itself can be generated. The organization may 
receive a rating that indicates the extent to which the 
organization's overall reporting is considered transparent. 
In one implementation, a rating of ''1'' would indicate that the 
organization's level of reporting about itself is excellent, 

15 whereas a rating of ''5" would indicate that the organization's 
level of reporting is very poor and that significant 
improvement is recommended. Ratings of ''2," ''3" or "M" would 
indicate levels of reporting that fall somewhere between the 
high and low ratings. 

20 In addition, a total score for each performance measure 

can be calculated. The analysis results can include the 
organization's total score for each category and each element 
in the framework, and the total scores can be compared to the 
corresponding highest possible scores. 

25 FIG. 6 illustrates one technique for calculating the 

organization's actual total score for a performance measure 
and the maximum possible score for that performance measure. 
In the table of FIG. 6, PMx refers to the x^^ performance 
measure and Z indicates the possible number of points awarded. 

30 Wx indicates the weighting for the x^^ performance measure. 
Typically, Wx is assigned a value of 1. However, different 
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values may be assigned so that different performance measures 
carry a different weight in the overall calculations. 

In some implementations, different sources of reporting 
by the organization may receive different weights. For 
5 example, if some sources tend to be more important in a 
particular industry, those sources could be weighted more 
heavily when evaluating an organization in that industry. 
Similarly, certain elements in the framework may be weighted 
more heavily if those elements are more significant for the 
10 specific industry to which the organization to be evaluated 
belongs . 

A total score for a particular element in the framework 
can be obtained by calculating the sum of the total scores for 
each of the performance measures in that element. Similarly, 
15 a total score for a particular category can be obtained by 
calculating the sum of the total scores for each element in 
that category. Comparisons of the organization's actual 
scores to the maximum possible scores can be calculated as 
well . 

20 The organization's score can be presented alone, compared 

to previously determined best or recommended practices, or to 
a peer group of one or more companies. In general, the 
assessment of the organization' s level of reporting about 
itself can include a comparison to some pre-selected criteria. 

25 The results can be presented in various formats including 

charts or radar diagrams. The user of the evaluation tool can 
select the particular format in which the results are to be 
displayed. For example, FIG. 7 illustrates a radar diagram 
that plots the score for an organization around each element 

30 of the framework. Such diagrams can be generated automatically 
to display the organization's score against a peer group for 
all three questionnaires or individually by questionnaire. 
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The peer group can be selected, for example, based on 
industry, geography or market capitalization. The various 
formats summarize the effectiveness of the organization's 
communications about itself. 
5 Various features of the system can be implemented in 

hardware, software, or a combination of hardware and software. 
For example, some features of the system can be implemented in 
computer programs executing on programmable computers. Each 
program can be implemented in a high level procedural or 

10 object-oriented programming language to communicate with a 

computer system. Furthermore, each such computer program can 
be stored on a storage medium such as read-only-memory (ROM) 
readable by a general or special purpose programmable computer 
or processor, for configuring and operating the computer to 

15 perform the functions described above. 

Various options can be made available to a user through 
the use of drop-down menus or graphical user interfaces to 
allow the user to select, for example, the desired 
questionnaires and criteria against which the organization is 

20 to be assessed. 

The foregoing implementations, including details of the 
questionnaires, the communication types and points awarded, as 
well as the calculations used to obtain total scores, are 
intended as examples only. Other implementations are within 

25 the scope of the claims. 
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